33 research outputs found
One Model is All You Need: Multi-Task Learning Enables Simultaneous Histology Image Segmentation and Classification
The recent surge in performance for image analysis of digitised pathology
slides can largely be attributed to the advance of deep learning. Deep models
can be used to initially localise various structures in the tissue and hence
facilitate the extraction of interpretable features for biomarker discovery.
However, these models are typically trained for a single task and therefore
scale poorly as we wish to adapt the model for an increasing number of
different tasks. Also, supervised deep learning models are very data hungry and
therefore rely on large amounts of training data to perform well. In this paper
we present a multi-task learning approach for segmentation and classification
of nuclei, glands, lumen and different tissue regions that leverages data from
multiple independent data sources. While ensuring that our tasks are aligned by
the same tissue type and resolution, we enable simultaneous prediction with a
single network. As a result of feature sharing, we also show that the learned
representation can be used to improve downstream tasks, including nuclear
classification and signet ring cell detection. As part of this work, we use a
large dataset consisting of over 600K objects for segmentation and 440K patches
for classification and make the data publicly available. We use our approach to
process the colorectal subset of TCGA, consisting of 599 whole-slide images, to
localise 377 million, 900K and 2.1 million nuclei, glands and lumen
respectively. We make this resource available to remove a major barrier in the
development of explainable models for computational pathology
Mitosis Detection, Fast and Slow: Robust and Efficient Detection of Mitotic Figures
Counting of mitotic figures is a fundamental step in grading and
prognostication of several cancers. However, manual mitosis counting is tedious
and time-consuming. In addition, variation in the appearance of mitotic figures
causes a high degree of discordance among pathologists. With advances in deep
learning models, several automatic mitosis detection algorithms have been
proposed but they are sensitive to {\em domain shift} often seen in histology
images. We propose a robust and efficient two-stage mitosis detection
framework, which comprises mitosis candidate segmentation ({\em Detecting
Fast}) and candidate refinement ({\em Detecting Slow}) stages. The proposed
candidate segmentation model, termed \textit{EUNet}, is fast and accurate due
to its architectural design. EUNet can precisely segment candidates at a lower
resolution to considerably speed up candidate detection. Candidates are then
refined using a deeper classifier network, EfficientNet-B7, in the second
stage. We make sure both stages are robust against domain shift by
incorporating domain generalization methods. We demonstrate state-of-the-art
performance and generalizability of the proposed model on the three largest
publicly available mitosis datasets, winning the two mitosis domain
generalization challenge contests (MIDOG21 and MIDOG22). Finally, we showcase
the utility of the proposed algorithm by processing the TCGA breast cancer
cohort (1,125 whole-slide images) to generate and release a repository of more
than 620K mitotic figures.Comment: Extended version of the work done for MIDOG challenge submissio
Social network analysis of cell networks improves deep learning for prediction of molecular pathways and key mutations in colorectal cancer
Colorectal cancer (CRC) is a primary global health concern, and identifying the molecular pathways, genetic subtypes, and mutations associated with CRC is crucial for precision medicine. However, traditional measurement techniques such as gene sequencing are costly and time-consuming, while most deep learning methods proposed for this task lack interpretability. This study offers a new approach to enhance the state-of-the-art deep learning methods for molecular pathways and key mutation prediction by incorporating cell network information. We build cell graphs with nuclei as nodes and nuclei connections as edges of the network and leverage Social Network Analysis (SNA) measures to extract abstract, perceivable, and interpretable features that explicitly describe the cell network characteristics in an image. Our approach does not rely on precise nuclei segmentation or feature extraction, is computationally efficient, and is easily scalable. In this study, we utilize the TCGA-CRC-DX dataset, comprising 499 patients and 502 diagnostic slides from primary colorectal tumours, sourced from 36 distinct medical centres in the United States. By incorporating the SNA features alongside deep features in two multiple instance learning frameworks, we demonstrate improved performance for chromosomal instability (CIN), hypermutated tumour (HM), TP53 gene, BRAF gene, and Microsatellite instability (MSI) status prediction tasks (2.4%–4% and 7–8.8% improvement in AUROC and AUPRC on average). Additionally, our method achieves outstanding performance on MSI prediction in an external PAIP dataset (99% AUROC and 98% AUPRC), demonstrating its generalizability. Our findings highlight the discrimination power of SNA features and how they can be beneficial to deep learning models’ performance and provide insights into the correlation of cell network profiles with molecular pathways and key mutations
NuClick : a deep learning framework for interactive segmentation of microscopic images
Object segmentation is an important step in the workflow of computational pathology. Deep learning based models generally require large amount of labeled data for precise and reliable prediction. However, collecting labeled data is expensive because it often requires expert knowledge, particularly in medical imaging domain where labels are the result of a time-consuming analysis made by one or more human experts. As nuclei, cells and glands are fundamental objects for downstream analysis in computational pathology/cytology, in this paper we propose NuClick, a CNN-based approach to speed up collecting annotations for these objects requiring minimum interaction from the annotator. We show that for nuclei and cells in histology and cytology images, one click inside each object is enough for NuClick to yield a precise annotation. For multicellular structures such as glands, we propose a novel approach to provide the NuClick with a squiggle as a guiding signal, enabling it to segment the glandular boundaries. These supervisory signals are fed to the network as auxiliary inputs along with RGB channels. With detailed experiments, we show that NuClick is applicable to a wide range of object scales, robust against variations in the user input, adaptable to new domains, and delivers reliable annotations. An instance segmentation model trained on masks generated by NuClick achieved the first rank in LYON19 challenge. As exemplar outputs of our framework, we are releasing two datasets: 1) a dataset of lymphocyte annotations within IHC images, and 2) a dataset of segmented WBCs in blood smear images
Domain Generalization in Computational Pathology: Survey and Guidelines
Deep learning models have exhibited exceptional effectiveness in
Computational Pathology (CPath) by tackling intricate tasks across an array of
histology image analysis applications. Nevertheless, the presence of
out-of-distribution data (stemming from a multitude of sources such as
disparate imaging devices and diverse tissue preparation methods) can cause
\emph{domain shift} (DS). DS decreases the generalization of trained models to
unseen datasets with slightly different data distributions, prompting the need
for innovative \emph{domain generalization} (DG) solutions. Recognizing the
potential of DG methods to significantly influence diagnostic and prognostic
models in cancer studies and clinical practice, we present this survey along
with guidelines on achieving DG in CPath. We rigorously define various DS
types, systematically review and categorize existing DG approaches and
resources in CPath, and provide insights into their advantages, limitations,
and applicability. We also conduct thorough benchmarking experiments with 28
cutting-edge DG algorithms to address a complex DG problem. Our findings
suggest that careful experiment design and CPath-specific Stain Augmentation
technique can be very effective. However, there is no one-size-fits-all
solution for DG in CPath. Therefore, we establish clear guidelines for
detecting and managing DS depending on different scenarios. While most of the
concepts, guidelines, and recommendations are given for applications in CPath,
we believe that they are applicable to most medical image analysis tasks as
well.Comment: Extended Versio